Tight Bounds for $\ell_p$ Oblivious Subspace Embeddings
نویسندگان
چکیده
An lp oblivious subspace embedding is a distribution over r × n matrices Π such that for any fixed n× d matrix A, Pr Π [for all x, ‖Ax‖p ≤ ‖ΠAx‖p ≤ κ‖Ax‖p] ≥ 9/10, where r is the dimension of the embedding, κ is the distortion of the embedding, and for an n-dimensional vector y, ‖y‖p = ( ∑n i=1 |yi|) 1/p is the lp-norm. Another important property is the sparsity of Π, that is, the maximum number of non-zero entries per column, as this determines the running time of computing Π ·A. While for p = 2 there are nearly optimal tradeoffs in terms of the dimension, distortion, and sparisty, for the important case of 1 ≤ p < 2, much less was known. In this paper we obtain nearly optimal tradeoffs for lp oblivious subspace embeddings for every 1 ≤ p < 2. Our main results are as follows: 1. We show for every 1 ≤ p < 2, any oblivious subspace embedding with dimension r has distortion κ = Ω ( 1 ( 1 d ) 1/p ·log2/p r+( r n ) 1/p−1/2 ) . When r = poly(d) ≪ n in applications, this gives a κ = Ω(d log d) lower bound, and shows the oblivious subspace embedding of Sohler and Woodruff (STOC, 2011) for p = 1 and the oblivious subspace embedding of Meng and Mahoney (STOC, 2013) for 1 < p < 2 are optimal up to poly(log(d)) factors. 2. We give sparse oblivious subspace embeddings for every 1 ≤ p < 2 which are optimal in dimension and distortion, up to poly(log d) factors. Importantly for p = 1, we achieve r = O(d log d), κ = O(d log d) and s = O(log d) non-zero entries per column. The best previous construction with s ≤ poly(log d) is due to Woodruff and Zhang (COLT, 2013), giving κ = Ω(dpoly(log d)) or κ = Ω(d √ log n · poly(log d)) and r ≥ d · poly(log d); in contrast our r = O(d log d) and κ = O(d log d) are optimal up to poly(log(d)) factors even for dense matrices. We also give (1) nearly-optimal lp-subspace embeddings with an expected 1 + ǫ number of non-zero entries per column for arbitrarily small ǫ > 0, and (2) the first oblivious subspace embeddings for 1 ≤ p < 2 with O(1)-distortion and dimension independent of n. Oblivious subspace embeddings are crucial for distributed and streaming environments, as well as entrywise lp low rank approximation. Our results give improved algorithms for these applications.
منابع مشابه
Subspace Embeddings and \(\ell_p\)-Regression Using Exponential Random Variables
Oblivious low-distortion subspace embeddings are a crucial building block for numerical linear algebra problems. We show for any real p, 1 ≤ p < ∞, given a matrix M ∈ R with n ≫ d, with constant probability we can choose a matrix Π with max(1, n)poly(d) rows and n columns so that simultaneously for all x ∈ R, ‖Mx‖p ≤ ‖ΠMx‖∞ ≤ poly(d)‖Mx‖p. Importantly, ΠM can be computed in the optimal O(nnz(M)...
متن کاملNearly Tight Oblivious Subspace Embeddings by Trace Inequalities
We present a new analysis of sparse oblivious subspace embeddings, based on the ”matrix Chernoff” technique. These are probability distributions over (relatively) sparse matrices such that for any d-dimensional subspace of R, the norms of all vectors in the subspace are simultaneously approximately preserved by the embedding with high probability–typically with parameters depending on d but not...
متن کاملLower Bounds for Oblivious Subspace Embeddings
An oblivious subspace embedding (OSE) for some ε, δ ∈ (0, 1/3) and d ≤ m ≤ n is a distribution D over Rm×n such that for any linear subspace W ⊂ Rn of dimension d, P Π∼D (∀x ∈W, (1− ε)‖x‖2 ≤ ‖Πx‖2 ≤ (1 + ε)‖x‖2) ≥ 1− δ. We prove that any OSE with δ < 1/3 must have m = Ω((d + log(1/δ))/ε2), which is optimal. Furthermore, if every Π in the support of D is sparse, having at most s non-zero entries...
متن کاملTight Bounds for Sketching the Operator Norm, Schatten Norms, and Subspace Embeddings
We consider the following oblivious sketching problem: given ∈ (0, 1/3) and n ≥ d/ 2, design a distribution D over Rk×nd and a function f : R × R → R, so that for any n× d matrix A, Pr S∼D [(1− )‖A‖op ≤ f(S(A), S) ≤ (1 + )‖A‖op] ≥ 2/3, where ‖A‖op = supx:‖x‖2=1 ‖Ax‖2 is the operator norm of A and S(A) denotes S ·A, interpreting A as a vector in R. We show a tight lower bound of k = Ω(d2/ 2) for...
متن کاملSubspace Embeddings and ℓp-Regression Using Exponential Random Variables
Oblivious low-distortion subspace embeddings are a crucial building block for numerical linear algebra problems. We show for any real p, 1 ≤ p <∞, given a matrix M ∈ Rn×d with n d, with constant probability we can choose a matrix Π with max(1, n1−2/p)poly(d) rows and n columns so that simultaneously for all x ∈ R, ‖Mx‖p ≤ ‖ΠMx‖∞ ≤ poly(d)‖Mx‖p. Importantly, ΠM can be computed in the optimal O(n...
متن کامل